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Abstract — Assuming iterative decoding for binary erasure 
channels (BECs), a novel tree-based technique for upper bound- 
ing the bit error rates (BERs) of arbitrary, finite low-density 
parity-check (LDPC) codes is provided and the resulting bound 
can be evaluated for all operating erasure probabilities, including 
both the waterfall and the error floor regions. This upper bound 
can also be viewed as a narrowing search of stopping sets, which 
is an approach different from the stopping set enumeration used 
for lower bounding the error floor. When combined with optimal 
leaf -finding modules, this upper bound is guaranteed to be tight in 
terms of the asymptotic order. The Boolean framework proposed 
herein further admits a composite search for even tighter results. 
For comparison, a refinement of the algorithm is capable of 
exhausting all stopping sets of size < 13 for irregular LDPC 
codes of length n « 500, which requires ( 5 1 ° 3 ) « 1.67 x 10 25 trials 
if a brute force approach is taken. These experiments indicate 
that this upper bound can be used both as an analytical tool and 
as a deterministic worst-performance (error floor) guarantee, the 
latter of which is crucial to optimizing LDPC codes for extremely 
low BER applications, e.g., optical/satellite communications. 

I. Introduction 

The bit error rate (BER) curve of any fixed, finite, low- 
density parity-check (LDPC) code on binary erasure channels 
(BECs) is completely determined by its stopping set distri- 
bution. Due to the prohibitive cost of computing the entire 
stopping set distribution [1], in practice, the waterfall threshold 
of the BER is generally approximated by the density evolution 
and pinpointed by the Monte-Carlo simulation, while the error 
floor is lower bounded by semi-exhaustively identifying the 
dominant stopping sets [2] or by importance sampling [3]. 
Even computing the size of the minimum stopping sets has 
been proved to be an NP-hard problem [4], which further 
shows the difficulty of constructing the entire stopping set 
distribution. Other research directions related to the finite code 
performance include [5] and [6] on the average performance 
of finite code ensembles on BECs and its scaling law, and a 
physics-based asymptotic approximation for Gaussian chan- 
nels [7]. 

In this paper, only BECs will be considered. We focus on 
upper bounding the BER curves of arbitrary, fixed, finite parity 
check codes under iterative decoding, and the frame error rate 
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Fig. 1. A simple parity check code. 

(FER) will be treated as a special case. Experiments are con- 
ducted for the cases n = 24, 50, 72, 144, which demonstrate 
the superior efficiency of the proposed algorithm. Application 
of this bound to finite code optimization is deferred to a 
companion paper. 

II. Boolean Expressions with Nested Structures 

Without loss of generality, we assume the all-zero codeword 
is transmitted for notational simplicity. 

For BECs, a decoding algorithm for bit Xi E {0, e}, i G 
[l,n] is equivalent to a function gi : {0, e} n i— > {0, e}, where 
"e" represents an erased bit and n is the codeword length. 
In this paper, fij, Vz € [l,n], is used to denote the iterative 
decoder for bit Xi after I iterations. If we further rename the 
element "e" by "1," foj becomes a Boolean function, and the 
BER for bit Xi after I iterations is simply pij = Ej/^;}. 
Another advantage of this conversion is that the decoding 
operation at the variable node then becomes the binary 
AND operation, and the operation at the parity check node 
becomes "+", the binary OR operation. 

For example, suppose we further use fi->j,i to represent the 
message from variable node i to check node j during the Z-th 
iteration, and consider the simple code described in Fig. 1. 
The iterative decoders f2,i, I G {1, 2}, for bit X2 then become 

/ 2 ,i = x 2 {xi + x 3 )(x 4 + x e ) 

h,2 = 212 (»l(/5— 4,1 +/6— 4,l) +a!3(/4— 3,1 +/5— 3,l)) 

■ (Z4(/3— 3,1 + /s— 3,l) + Z 6 (/l-,4,l + /s— 4,l)) 

= x 2 {xi(x 5 + x 6 ) + x 3 (x 4 + x 5 )) 

■ (x 4 (x 3 + x 5 ) + x 6 (xi + x 5 )) . (1) 

The final decoder of bit x 2 is f 2 ■= limj_»oo fa,u and in this 
example, f 2 = f 2 2 . Although (1) admits a beautiful nested 
structure, the repeated appearance of many Boolean input 
variables, also known as short "cycles," poses a great challenge 



to the evaluation of the BER p 2 = E{/2}. One solution is to 
first simplify (1) by expanding the nested structure into a sum- 
product Boolean expression [1]: 

f 2 = XlX 2 X 4 X 5 + X 2 X 3 X 4 + 2:2X3X5X6 . (2) 

E{/2} can then be evaluated by the inclusion-exclusion prin- 
ciple: p 2 = e 3 + 2e 4 — 2e 5 , where e is the erasure probability. 

It can be proved that each product term corresponds to 
an irreducible stopping set (IRSS) and vice versa. Instead of 
constructing the exact expression of if only a small subset 
of these IRSSs is identified, say "x 2 x 3 X4," then a lower bound 
E{./lb,2} = e 3 can be obtained, where 

/lb, 2 = X2X3X4 < f 2 , and E{f LBi2 } = e 3 < E{/ 2 }. 

The major challenge of this approach is to ensure that all 
IRSSs of the minimum weight/order are exhausted. Further- 
more, even when all IRSSs of the minimum weight are 
exhausted, this lower bound is tight only in the high signal-to- 
noise ratio (SNR) regime. Whether the SNR of interest is high 
enough can only be determined by Monte-Carlo simulations 
and by extrapolating the waterfall region. 

An upper bound can be constructed by iteratively computing 
the sum-product form of / 2 ,; +i from that of f 2 .i . To counter- 
act the exponential growth rate of the number of product terms, 
during each iteration, we can "relax" and "merge" some of the 
product terms so that the complexity is kept manageable [1]. 
For example, / 2 ,2 in (2) can be relaxed and merged as follows. 

/2,2 = X1X2X4X5 + X2X3X4 + X2X3X5X6 
< lx 2 X 4 l + X 2 1X 4 + X2X 3 X 5 X 6 
= X 2 X 4 + X2X 3 X 5 X 6 , 

so that the number of product terms is reduced to two. 
Nonetheless, the minimal number of product terms required to 
generate a tight upper bound grows very fast and good/tight 
results were reported only for the cases n < 20. In contrast, we 
construct an efficient upper bound UB^ > E{fi] by preserving 
much of the nested structure, so that tight upper bounds can 
be obtained for n — 100-300. Furthermore, the tightness of 
our bound can be verified with ease, which was absent in the 
previous approach. Combined with the lower bound E{/lb,i}, 
the finite code performance can be efficiently bracketed for the 
first time. 

III. An Upper Bound Based on Tree-Trimming 

Two fundamental observations can be proved as follows. 

Observation 1: All /j's are monotonic functions w.r.t. all 
input variables. Namely, fi\ Xj= o < fi\ Xj =i for all i,j <E [1, n], 
which separates /j's from "arbitrary" Boolean functions. (Here 
we use the point-wise ordering such that / < g iff /(x) < 
g(x) for all binary vectors x.) 

Observation 2: The correlation coefficient between any 
pair of fi and fj is always non-negative, i.e., E{/j ■ fj} > 
E{/i}E{£}. 

In this section, we assume that /„ = g ■ h or f c = g + h for 
variable or check node operations respectively. Define x g C 




(a) Original (b) Relaxation 

Fig. 2. Rule 1: A simple relaxation for check nodes. 
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Fig. 3. Rule 2: A pivoting rale for variable nodes. 

{xi, • • • , x n } as the set of input variables upon which the 
Boolean function g depends, e.g., if g = x\x 2 + X7, then 
x g = {xi,x 2 ,x 7 }. Similary, we can define x/j. 

A. Rule 

If x g n x/j = 0, namely, there is no repeated node 
in the input arguments, then 

E{/„} = E{g}E{h} 

E{f c } = E{g} + E{h}-E{g}E{h}. 

B. Rule 1 - A Simple Relaxation 

Suppose x 9 nx),^ 0, namely, there are repeated nodes in 
the input arguments. By Observation 2 and Rule 0, we have 

E{/J = E{g} + E{h} - E{g ■ h} 

< E{g} + E{h}-E{g}E{h}. (3) 

The above rule suggests that when the incoming messages 
of a check node are dependent, the error probability of the 
outgoing message can be upper bounded by assuming the 
incoming ones are independent. Furthermore, Rule 1 does not 
change the order of error probability, as can be seen in (3), 
but only modifies the multiplicity term. Due to the random- 
like interconnection within the code graph, for most cases, g 
and h are "nearly independent" and the multiplicity loss is not 
significant. The realization of Rule 1 is illustrated in Fig. 2, 
in which we assume that x io is the repeated node. 

C. Rule 2 - The Pivoting Rule 

Consider the simplest case in which x 9 n Xh = {xi }. By 
Observation 1, we have 

fv = 9v\xi =0 " hv\xi o =0 

+Xi ■ 9v |x i0 =i • h v \ XiQ= i. (4) 

The realization of the above equation is demonstrated in Fig. 3. 
Once the tree in Fig. 3(a) is transformed to Fig. 3(b), all 
messages entering the variable nodes become independent 



Algorithm 1 A tree-based method to upper bound p,. 

Initialization: Let T be a tree containing only the target xt variable node. 
1: repeat 

2: Find the next leaf variable node, say xj . 

3: if there exists another non-leaf Xj variable node in T then 

4: if the youngest common ancestor of the leaf Xj and any other existing 

non-leaf Xj, denoted as yca(xj), is a check node then 
5: As suggested by Rule 1, the new leaf node Xj can be directly 

included as if there are no other Xj's in T. 
6: else if yea (x 3 ■ ) is a variable node then 

7: As suggested by Rule 2, a pivoting construction involving tree 

duplication is initiated, which is illustrated in Fig. 3. 
8: end if 
9: end if 

10: Construct the immediate check node children and variable node grand 
children of Xj as in the support tree of the corresponding Tanner graph. 
1 1 : until the size of T exceeds the preset limit. 
12: Hardwire all remaining leaf nodes to 1. 

13: UBi is evaluated by invoking Rules and 1. Namely, all incoming edges 
are blindly assumed to be independent. 



since g v \ XiQ= o and h v \ Xia=0 then share no repeated input 
variables. By reapplying Rules and 1, the output E{f v } is 
upper bounded by 

E{/4 < E{f v \x lQ =0} + E{x lo }E{f v \x lo = l} 
-E{f v \x io = 0}E{x io }E{f v \x io = 1}, 

where for all b G {0,1}, 

E{fv\xt = b} = E{g v \x t0 = b}E{h v \x io = b}. 

Note: the pivoting rule (4) does not incur any performance 
loss. The actual loss during this step is the multiplicity loss 
resulted from reapplying Rule 1 . Therefore, Rule 2 preserves 
the asymptotic order of E{f v } as does Rule 1. 

D. The Algorithm 

Rules to 2 are designed to upper bound the expectation of 
single operations with zero or a single overlapped node. Once 
carefully concatenated, they can be used to construct UB^ for 
the infinite tree with many repeated nodes, while preserving 
most of the nested structure. 

Theorem 1: The concatenation in Algorithm 1 is guaranteed 
to find an upper bound UB^ for pi of the infinite tree. 

The proof of Theorem 1 involves the graph theoretic prop- 
erties of yca(x., ) and an incremental tree-revealing argument. 
Some other properties of Algorithm 1 are listed as follows. 

> The only computationally expensive step is when Rule 2 
is invoked, which, in the worst case, may double the tree 
size and thus reduces the efficiency of this algorithm. 

• Rule 1, being the only relaxation rule, saves much com- 
putational cost by ignoring repeated nodes. 

• Once the tree construction is completed, evaluating 
UBi for any e G [0, 1] is efficient with complexity 
O (|T| log(|TQ), where |T| is the size of T. 

« The preset size limit of T provides a tradeoff between 
computational resources and the tightness of the resulting 
UBi. One can terminate the program early before the 
tightest results are obtained, as long as the intermediate 
results have met the evaluation/design requirements. 



This tree-based approach corresponds to a narrowing search 
of stopping sets. By denoting fr,t as the corresponding 
Boolean function of the tree 1 T at time t, we have 

Theorem 2 (A Narrowing Search): Let 

X t := {(a*, • • • , x n ) G {0, 1}" : f Ttt ( Xl , ■ ■■,*„) = 1}. 
We then have 

{all stopping sets} C X t+ i C X t ,Vi G N. 

IV. Performance and Related Topics 

A. The Leaf-Finding Module 

The tightness of UB^ in Algorithm 1 depends heavily on 
the leaf-finding (LF) module invoked in Line 2. A properly 
designed LF module is capable of increasing the asymptotic 
order of UBj by +1 to +3. The ultimate benefit of an optimal 
LF module is stated in the following theorem. 

Theorem 3 (The Optimal LF Module): Following the nota- 
tion in Theorem 2, with an "optimal" LF module, we have 

{all stopping sets} = lim X t . 

t — >oo 

Corollary 1 ( Order Tightness of Algorithm 1 ): When 
combined with an optimal LF module, the UB^ computed 
by Algorithm 1 is tight in terms of the asymptotic order. 
Namely, 3C > such that < C for all e G (0, 1]. 

In [4], determining whether a fixed LDPC codes contains 
a stopping set of size < t is proved to be NP-hard. By 
Theorem 2, a straightforward choice of the LF module, and 
the complexity analysis of Algorithm 1, we have 

Theorem 4: Deciding whether the stopping distance is < t 
is fixed-parameter tractable when t is fixed. 

For all our experiments, an efficient approximation of the 
optimal LF module, motivated by the proof of Theorem 3, 
is adopted. With reasonable computational resources, Algo- 
rithm 1 is capable of constructing asymptotically tight UBs 
for LDPC codes of n < 100. A composite approach will be 
introduced later, which further extends the application range 
to n < 300. 

B. Confirming the Tightness of UBi 

To this end, after each time t, we first exhaustively enu- 
merate the elements of minimal weight in X t and denote the 
collection of them as X m i„. 

Corollary 2 (Tightness Confirmation): If 3x G X m i„ that 
is a stopping set, then UBi is tight in terms of the asymptotic 
order. 

Corollary 3 (The Tight Upper and Lower Bound Pair): 
Let X mi „ 55 C X mi „ denote the collection of all elements 
x G X„ lin that are also stopping sets. Then X mi „ 5 S exhausts 
the stopping sets of the minimal weight, and can be used 
to derive a lower bound E{/ LB ,i} that is tight in both the 
asymptotic order and the multiplicity. This exhaustive lower 
bound was not found in any existing papers. 

'Algorithm 1 consists of the tree construction stage and the upper bound 
computing stage (Line 13). In this paper, fq- ; t ■ {0, l} n i-> {0, 1} is defined 
on the constructed tree, which will then be used on evaluating UB^. 
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TABLE I 

PERFORMANCE STATISTICS: "Num. bits" is the number of bits with the specified asymptotic order, "order*" is the num. bits with UBs tight only in the order. "+ multi*" is the 
num. bits with UBs tight both in the order and in the multiplicity, "order >" is the num. bits with a UB of the specified order while no bracketing lower bound can be established. 



C. BER vs. FER 

The above discussion has focused on providing UBi for a 
pre-selected target bit Xi. Bounds for the average BER can be 
easily obtained by taking averages over bounds for individual 
bits. An equally interesting problem is bounding the FER, 
which can be converted to the BER as follows. Introduce an 
auxiliary variable and check node pair (xq, Vo), such that the 
the new variable node x is punctured and the new check 
node j/o is connected to all n + 1 variable nodes from Xo to 
x n . The FER of the original code now equals the BER po of 
variable node x and can be bounded by Algorithm 1. Since 
the FER depends only on the worst bit performance, it is easier 
to construct tight UB for the FER than for the BER. On the 
other hand, UBi provides detailed performance prediction for 
each individual bit, which is of great use during code analysis. 

D. A Composite Approach 

The expectation E{fi} can be further decomposed as 

M 

E{/ i } = ^E{^}E{/ i K-}, 

where Aj's are M events partitioning the sample space. For 
example, we can define a collection of non-uniform Aj 's by 

Ai = {x = 0} 

Ai = {x = l,x 7 = 0} 

A 3 = {x = l,x 7 = 1}. 

Since for any j, fi\^ is simply another finite code with a 
modified Tanner graph, Algorithm 1 can be applied to each 
fi\Aj respectively and different UB i:) > Ej/jl^} will be 
obtained. A composite upper bound is now constructed by 

M 

C-UB, = ElAlUBu > E{fi} = ^ 

3 = 1 

In general, C-UB is able to produce bounds that are +1 or +2 
in the asymptotic order and pushes the application range to 
n < 300. The efficiency of C-UB relies on the design of the 
non-uniform partition {Aj}. 

E. Performance 

1) The (23,12) Binary Golay Code: The standard parity 
check matrix of the Golay code is considered. Fig. 4 compares 
the upper bound (UB), the composite upper bound (C-UB), 
the Monte-Carlo simulation (MC-S), and the side product, the 



tight lower bound (LB), on bits 0, 5, and 20. As illustrated, C- 
UB and LB tightly bracket the MC-S results, which shows that 
our UB and C-UB are capable of decoupling even non-sparse 
Tanner graphs with plenty of cycles. 

2) A (3,6) LDPC Code with n = 50: A (3,6) LDPC code 
with n = 50 is randomly generated, and the UB, the C-UB, the 
MC-S, and the tight LB are performed on bits 0, 26, and 19, as 
plotted in Fig. 5, and the statistics of all 50 bits are provided 
in TABLE 1(a). Our UB is tight in the asymptotic order for 
all bits while 34 bits are tight in multiplicity. Among the 16 
bits not tight in multiplicity, 11 bits are within a factor of 
three of the actual multiplicity. In contrast with the Golay 
code example, the tight performance can be attributed to the 
sparse connectivity of the corresponding Tanner graph. As can 
be seen in Fig. 5(c), the C-UB possesses the greatest advantage 
over those UBs without tight multiplicity. The C-UB and the 
LB again tightly bracket the asymptotic performance. 

3) A (3,6) LDPC Code with n = 72: The UB, the C-UB, 
the MC-S, and the tight LB are applied to bits 41, 25, and 60, 
as plotted in Fig. 6 and the statistics are in TABLE 1(b). Almost 
all asymptotic orders can be captured by the UB with only two 
exception bits. Both of the exception bits are of order 8, which 
is computed by applying the C-UB to each bit respectively. 

4) (3,6) LDPC Codes with n = 144: Complete statistics are 
presented in TABLE 1(c), and we start to see many examples 
(101 out of 144 bits) in which our simple UB is not able to 
capture the asymptotic order. For those bits, we have to resort 
to the C-UB for tighter results. It is worth noting that the 
simple UB is able to identify some bits with order 9, which 
requires ( 1 g 4 ) = 5.7 x 10 13 trials if a brute force method is 
employed. Furthermore, all stopping sets of size < 7 have been 
identified, which shows that Algorithm 1 is able to generate 
tight UBs when only FERs are considered. Among all our 
experiments, many of which are not reported herein, the most 
computationally friendly case is when considering FERs for 
irregular codes with many degree 2 variable nodes, which are 
one of the most important subjects of current research. In these 
scenarios, all stopping sets of size < 13 have been identified 
for non-trivial irregular codes with n — 576, which evidences 
the superior efficiency of the proposed algorithm. 

V. Conclusion & Future Directions 

A new technique upper bounding the BER of any finite code 
on BECs has been established, which, to our knowledge, is the 
first algorithmic result guaranteeing finite code performance 
while admitting efficient implementation. Preserving much 




Fig. 4. Comparisons among the upper bound (UB), the composite upper bound (C-UB), the Monte-Carlo simulation (MC-S), and the side product tight 
lower bound (LB) for bits 0, 5, and 20 of the (23,12) binary Golay code. The corresponding asymptotic (order, multiplicity) pairs obtained by UB, C-UB, 
and the actual BER is are V0: {(3, 10), (4, 75), (4, 75)}, V5: {(3, 15), (4, 50), (4, 45)}, and V20: {(4, 221), (4, 27), (4, 1)}. 
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Fig. 5. Comparisons among the UB, the C-UB, the MC-S, and the LB for bits 0, 26, and 19 of a randomly generated (3,6) LDPC code with n = 50. 
The asymptotic (order, multiplicity) pairs of the UB, the C-UB, and the actual BER are V0: {(4, 1), (4, 1), (4, 1)}, V26: {(6, 2), (6, 4), (6, 2)}, and V19: 
{(7, 135), (7, 10), (7, 5)}. 
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Fig. 6. Comparisons among the UB, the C-UB, the MC-S, and the LB for bits 41, 25, and 60 of a randomly generated (3,6) LDPC code with n = 
72. The asymptotic (order, multiplicity) pairs of the UB, the C-UB, and the actual BER are V41: {(4, 1), -, (4, 1)}, V25: {(7, 1), -, (7, 1)}, and V60: 
{(7, 19), (8, 431), (8, 11)}. 



of the decoding tree structure, this bound corresponds to a 
narrowing search of stopping sets. The asymptotic tightness 
of this technique has been proved, while the experiments 
demonstrate the inherent efficiency of this method for codes of 
moderate sizes n < 300. One major application of this upper 
bound is to design high rate codes with guaranteed asymptotic 
performance, and our results specify both the attainable low 
BER, e.g., 10~ 15 , and at what SNR it can be achieved. 
One further research direction is on extending the setting to 
binary symmetric channels with quantized belief propagation 
decoders such as Gallager's decoding algorithms A and B. 
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